The proliferation of the Internet of Things (IoT) and its cutting-edge AI-enabled applications (e.g., autonomous vehicles and smart industries) combine two paradigms: data-driven systems and their deployment on the edge. Usually, edge devices perform inferences to support latency-critical applications. In addition to the performance of these resource-constrained edge devices, their energy usage is a critical factor in adopting and deploying edge applications. Examples of such devices include Raspberry Pi (RPi), Intel Neural Compute Stick (INCS), NVIDIA Jetson nano (NJn), and Google Coral USB (GCU). Despite their adoption in edge deployment for AI inferences, there is no study on their performance and energy usage for informed decision-making on the device and model selection to meet the demands of applications. This study fills the gap by rigorously characterizing the performance of traditional, neural networks, and large language models on the above-edge devices. Specifically, we analyze trade-offs among model F1 score, inference time, inference power, and memory usage. Hardware and framework optimization, along with external parameter tuning of AI models, can balance between model performance and resource usage to realize practical edge AI deployments.

Mumbai: Early this month at the Intel AI Devcon 2018 in Bengaluru, a holographic avatar called Ella listened intently to composer Kevin Doucette playing notes on his synthesizer. When he paused, she began composing her own notes, complementing his music in real-time. Ella was learning about features such as tempo, scale and pitch from the music data that was being sent in real-time to an Intel Movidius Neural Compute Stick. Intel used a class of artificial neural networks, the recurrent neural network or RNN that depends on previous calculations to work on current ones, to perform this artificial intelligence (AI) task. This Neural Compute Stick is simply a case in point that Intel--a company which most people identify with central processing units (CPUs) inside personal computers (PCs), mobiles and servers--is widening its portfolio to stay in the AI race that has strong contenders including Nvidia, Microsoft, Google, Facebook, IBM, Amazon, Apple, Alibaba and Baidu.

In 1933, a chicken keeper and amateur photographer decided to find the culprit who was stealing his eggs. Since its inception, security cameras are everywhere nowadays, most of the claimed "smart ones" work by streaming videos back to a monitor or a server so as someone or some software can analyze video frames and hopefully find some useful information from them. They consume a large amount of network bandwidth and power to stream videos even though ten image frames are all we need to know who was stealing the eggs. They are also facing a dilemma of out of service when the network is unstable, images cannot be analyzed and the "smart" becomes "dumb". Edge computing is a network model which enables data processing occurs at the edge of the network where the camera is located, eliminating the need to send videos to a central server for processing.

Mumbai: Early this month at the Intel AI Devcon 2018 in Bengaluru, a holographic avatar called Ella listened intently to composer Kevin Doucette playing notes on his synthesizer. When he paused, she began composing her own notes, complementing his music in real-time. Ella was learning about features such as tempo, scale and pitch from the music data that was being sent in real-time to an Intel Movidius Neural Compute Stick. Intel used a class of artificial neural networks, the recurrent neural network or RNN that depends on previous calculations to work on current ones, to perform this artificial intelligence (AI) task. This Neural Compute Stick is simply a case in point that Intel--a company which most people identify with central processing units (CPUs) inside personal computers (PCs), mobiles and servers--is widening its portfolio to stay in the AI race that has strong contenders including Nvidia, Microsoft, Google, Facebook, IBM, Amazon, Apple, Alibaba and Baidu.

Aaeon has launched an "UP AI Edge" family of products that builds on a new Apollo Lake based "UP Core Plus" SBC with stacking AI companion boards based on the Movidius Myriad 2 or Intel Cyclone 10GX plus add-ons including a quad-GbE board and a camera. Aaeon Europe quickly met its modest $11K Kickstarter goal for the new UP AI Edge ecosystem, which builds on its UP board products and community. The centerpiece is a new UP Core Plus SBC, although the official, Ubuntu-equipped UP AI Edge development package uses the larger, more feature-rich UP Squared SBC. The Ubuntu stack also includes Intel's OpenVINO computer vision toolkit, which is optimized for the Myriad 2. Also available is the Arduino Create development environent, an Open CL/ Movidius Driver, Intel System Studio, and cloud connectors for Microsoft Azure, Amazon AWS, Google Cloud, and IBM Bluemix. You can also use the Neural Compute Stick SDK available for the Myriad 2 equipped Intel Movidius Neural Compute Stick for "rapid prototyping, validation and deployment of Deep Neural Network (DNN) inference applications at the edge," says Aaeon.

"Clean water and health care and school and food and tin roofs and cement floors, all of these things should constitute a set of basics that people must have as birthrights."1 Obtaining clean water is a critical problem for much of the world's population. Testing and confirming a clean water source typically requires expensive test equipment and manual analysis of the results. For regions in the world in which access to clean water is a continuing problem, simpler test methods could dramatically help prevent disease and save lives. To apply artificial intelligence (AI) techniques to evaluating the purity of water sources, Peter Ma, an Intel Software Innovator, developed an effective system for identifying bacteria using pattern recognition and machine learning.

Last year, Intel embedded its line of Myriad 2 chips into a USB stick that developers could plug into development boards or personal computers to experiment with artificial intelligence. The custom chips can accelerate neural networks used in image recognition and other tricky tasks that computers can be trained to tackle. On Tuesday, the company introduced a program to make it easier for developers to turn prototypes built with the Neural Compute Stick into security cameras, industrial sensors, and other production devices. Intel is partnering with Aaeon to offer a board called the A.I. Core, which serves as a sort of production version of the dongle. The A.I. Core board contains the same Myriad vision processing unit as the stick, allowing small companies and entrepreneurs to move to the production board without changing code.

Intel and Aaeon made it easier for hardware companies to build a machine learning accelerator into their products with today's launch of a new circuit board called the AI Core. That board contains a Movidius Myriad 2 Vision Processing Unit that speeds up execution of AI algorithms while only drawing around a watt of power. That's the same sort of capability hardware makers can get from the Movidius Neural Compute Stick, which looks like a somewhat bulky USB flash drive but offers AI acceleration. Ever since Intel released that hardware last year, it has picked up a following among hardware startups, makers, and developers interested in experimenting with AI. The stick is optimized for speeding up the execution of different types of machine learning algorithms, including convolutional neural networks, which are the backbone of many image recognition systems.

For the past couple of years, AI has turned from a "meh" kind of topic into one of the leading trends in almost every industry. Large corporations are buying AI-focused startups as fast as they can. At the same time, the market is witnessing an unprecedented amount of investments made in the area of AI. For example, Toyota raised a $100m. The technology has become so popular that you can find it in places where you expect it the least. One of the former Google engineers has even founded a religion that worships Artificial Intelligence. Zack Thoutt, a developer and a Game of Thrones fan, was so impatient to see the new season of the show that he decided to create a neural network that wrote all five chapters of the Fire and Ice saga.

Think only a large enterprise has the resources to deploy artificial intelligence technology? Intel aims to make AI more affordable and accessible, especially to smaller companies and entrepreneurs. Last month, the company introduced the Movidius Neural Compute Stick, which it billed as "the world's first USB-based deep learning inference kit and self-contained" AI accelerator. The $79 USB stick delivers "dedicated deep neural network processing capabilities to a wide range of host devices at the edge," Intel says. With the USB stick, Intel suggests that product developers, researchers and makers will be able to add AI capabilities to their devices and develop, tune and deploy AI-based applications far more easily.